MIXED OPERATORS IN COMPRESSED SENSING 
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Abstract. Applications of compressed sensing motivate the possibility of using different 
operators to encode and decode a signal of interest. Since it is clear that the operators 
cannot be too different, we can view the discrepancy between the two matrices as a pertur- 
bation. The stability of ^i-minimization and greedy algorithms to recover the signal in the 
presence of additive noise is by now well-known. Recently however, work has been done 
to analyze these methods with noise in the measurement matrix, which generates a multi- 
plicative noise term. This new framework of generalized perturbations (i.e., both additive 
and multiplicative noise) extends the prior work on stable signal recovery from incomplete 
and inaccurate measurements of Candes, Romberg and Tao using Basis Pursuit (BP), and 
of Needell and Tropp using Compressive Sampling Matching Pursuit (CoSaMP). We show, 
under reasonable assumptions, that the stability of the reconstructed signal by both BP 
and CoSaMP is limited by the noise level in the observation. Our analysis extends easily 
to arbitrary greedy methods. 



1. Introduction 

Compressed sensing refers to the problem of realizing a sparse, or nearly sparse, sig- 
nal from a small set of linear measurements. There are many applications of compressed 
sensing in engineering and the the sciences. Examples include biomedical imaging, x-ray 
crystallography, audio source separation, seismic exploration, radar and remote sensing, 
telecommunications, distributed and multi-sensor networks, machine learning, robotics and 
control, astronomy, surface metrology, coded aperture imaging, biosensing of DNA, and 
many more. See [5] for an extensive list of the latest literature. 

To precisely formulate the problem, we define an s-sparse sig nal x e C d to be one with s 
or fewer non-zero components, 

||sc||o = |supp(x)| <s<d. 

We apply a matrix A £ c mxa! to the signal and acquire measurements b = Ax. Often, we 
encounter additive noise so that the measurements become y = b + e = Ax + e, where e 
is an error or noise term usually assumed to have bounded energy ||e||2 < e. The field of 
compressed sensing has provided many recovery algorithms for sparse and nearly sparse 
signals, most with strong theoretical and numerical results. 

One major approach to sparse recovery is ^i-minimization or Basis Pursuit 00]. This 
method simply solves an optimization problem to recover the signal x, 

(1.1) min||z||i such that \\Az — y\\2 < e. 
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This problem can be solved using convex optimization techniques and is thus computation- 
ally feasible. Candes and Tao show in [4] that if the signal x is sparse and the measurement 
matrix A satisfies a certain quantitative property, then (jl.ip recovers the signal x exactly. 

Definition 1.1. A measurement matrix A satisfies the restricted isometry property (RIP) 
with parameters (s, 5) if for every s-sparse vector x, we have 

(1 - S)\\x\\l < \\Axg < (1 + 6)\\x\\l 

The parameter 5 is also referred to as the restricted isometry constant (RIC) of matrix A. 

It is now well known that many m x d matrices (e.g., random Gaussian, Bernoulli, and 
partial Fourier) satisfy the RIP with parameters (s, 5) when m = O(slogd), see |14(ll7j for 
details. It has been shown in [H [3] that if A satisfies the RIP with parameters (3s, 0.2), 
then (jl.ip recovers a signal x* that satisfies 

(1.2) \\x* - x\\ 2 < C ^ X ~ ^ +de 

Vs 

where x s denotes the vector consisting of the s largest components of x in magnitude. 

In [2] Candes sharpened this bound to work for matrices satisfying the RIP with param- 
eters (2s, \pl — 1), and later Foucart and Lai sharpened it to work for (2s, 0.4531) in |12j . 

Although the recovery guarantees provided by ^-minimization are strong, it requires 
methods of convex optimization which, although often quite efficient in practice, have a 
polynomial runtime. For this reason, much work in compressed sensing has been done to 
find faster methods. Many of these algorithms are greedy, and compute the (support of the) 
signal iteratively (see e.g., [181 (II El El OS E] ) • O ur analysis in this work focuses on Needell 
and Tropp's Compressive Sampling Matching Pursuit (CoSaMP) [To] , CoSaMP provides a 
fast runtime while also providing strong guarantees analogous to those of £i-minimization. 

The CoSaMP algorithm can be described as follows. We use the notation w\t and At 
to denote the vector w restricted to indices given by a set T, and the matrix A restricted 
to the the columns indexed by T, respectively. 

Compressive Sampling Matching Pursuit (CoSaMP) 
Input: Measurement matrix A, measurement vector y, sparsity level s 
Output: s-sparse reconstructed vector x = a 
Procedure: 

Initialize: Set a = 0, v = y, k = 0. Repeat the following steps and 

increment k until the halting criterion is true. 
Signal Proxy: Set u = A*v, Cl = supp(u2 S ) and merge the supports: 

T = ^Usupp(a fc - 1 ). 
Signal Estimation: Using least-squares, set w\t = A T y and w\t c = 0. 
Prune: To obtain the next approximation, set a k = w s . 
Sample Update: Update the current samples: v = y — Aa k . 



In [TS] it is shown that when the measurement matrix has a small RIC that CoSaMP 
approximately recovers arbitrary signals from noisy measurements. This is summarized by 
the following. 

Theorem 1.2 (CoSaMP |15|). Suppose that A is a measurement matrix with RIC 54 S < 0.1. 

Let y = Ax + e be a vector of samples of an arbitrary signal x, contaminated with arbitrary 
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noise. Then the algorithm CoSaMP produces an s -sparse approximation or that satisfies 
|M - x\\2 < C ■ (\\x - x s \\2 H — i=\\x - x s \U + llelU). 

2. Mixed Operators 

Applying the theories of compressed sensing to real-world problems raises the following 
question: what happens when the operator used to encode the signal is different from the 
operator used to decode the measurements? In many of the applications mentioned in 
Section 1 the sensing, or measurement, matrix A actually represents a system which the 
signal passes through. In other scenarios A represents some other physical phenomenon. 
For example, in screening for genetic disorders [10], the standard deviation of error in the 
sensing matrix is about 3%. This error is due to human handling when pipetting the DNA 
samples [9]. Whatever the setting may be, it is often the case that the true nature of 
this system is not known exactly. When this happens the system behavior is (perhaps 
unknowingly) approximated, or assumed to be represented, by a different matrix $. 

It is clear that the encoding and decoding operators, A and <!>, cannot be too different, 
but until recently there has been no analysis of the effect this difference has on reconstruc- 
tion error. In particular, the perturbations in the sensing matrices create a multiplicative 
noise term of the form (A — &)x. Herman and Strohmer first showed in p~3] that a noisy 
measurement matrix can be successfully used to recover a signal using £i-minimization. The 
natural question is whether this extends to the case of greedy algorithms. In this work, we 
consider the case of mixed operators in CoSaMP, and our results naturally apply to other 
greedy algorithms. 

In this analysis we will require examination of submatrices of certain matrices. To that 
end, we define 1 1 ^4 1 1 ^ to be the largest spectral norm over all s-column submatrices of A. 
Let 

» d -i f II „„a » 



(2.1) e%= 11 - -ffl and 

\2 



A-$\\ 2 S) ( S ) def VT+Is 

~ Ka ~ TT^Ts 



The first quantity is the relative perturbation of s-column submatrices of A with respect 
to to the spectral norm, and the second one bounds ratio of the extremal singular values of 
all s-column submatrices of A (see [13] for more details). We also need a measure of how 
"close" a signal x is to a sparse signal, and therefore define 

/n o\ dci 11^ -^S 1 1 2 -i n def 1 1 X 

(2-2) a s = — t — t and p s = — =■ — t— . 

\\xsh v s \\ x sh 

2.1. Mixed Operators in ^i-minimization. The work in [13] extended the previous re- 
sults in £i-minimization by generalizing the error term e which only accounted for additive 
noise. The new framework considers a total noise term £A,s,b which allows for both multi- 
plicative and additive noise. Theorem 12.11 b elow shows that the reconstruction error using 
^i-minimization is limited by this noise level. With regard to noise in operator A we see 

is) 

that the stability of the solution is a linear function of relative perturbations £a,s\ ■ 

Theorem 2.1 (Adapted from [13] . Thm. 2). Let x be an arbitrary signal with measurements 
b = Ax, corrupted with noise to form y = Ax + e. Assume the RIC for matrix A satisfies 

(2.3) 5 2s < - 1 
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and that general signal x satisfies 
(2.4) a s + p s < 



Set the total noise parameter 



/ (a) (a) , 

(2.5) e A , s ,fe := A (s) + e b 

where the relative perturbations ea = ^^J^ 2 , £& = jjfj|~> and^A = "j^f ■ i/ie solution 

z* to the BP problem ( fi.ij) e se£ to £a,s,6; using the decoding matrix 3> (instead of 
A) obeys 

Co 

(2.6) ||z*-x|| 2 < — F ||ai - x s ||i + Cie^ |S) 6 

V s 

/or some well-behaved constants Cq,C\. 

2.2. Mixed Operators in CoSaMP. We now turn to the case of mixed operators in 
greedy algorithms, and in particular CoSaMP. We will see that a result analogous to that 
of ^-minimization can be obtained in this well. Similar to condition (|2.4p above, 

we will again need for the signal to be well approximated by a sparse signal. To that end, 
we require that 

(2-7) a s + p s < — ^ 

2k a 

where a s and (3 S are defined in (|2.2p . Theorem 12. 21 below shows that under this assumption, 
the reconstruction error in CoSaMP is again limited by the tail of the signal and the 
observation noise. 

Theorem 2.2. Let A be a measurement matrix with RIC 

(2.8) 5 As < 1- 

(1 + 4 4S) ) 2 

Let x be an arbitrary signal with measurements b = Ax, corrupted with noise to form 
y = Ax + e. Let x" be the reconstruction from CoSaMP using decoding matrix (instead 
of A) on measurements y. Then if (|2.7p is satisfied, the estimation satisfies 

lb" - x|| 2 < C ■ (lb - x s \\ 2 + — =|b - x s |h + (ea s + £ (s) )||o|| 2 + ||e|| 2 
V y/s 

where e = \\A — <&|| 2 and = \\A - <3?|| 2 ^. 

Applying Theorem 12.21 to the sparse case, we immediately have the following corollary. 

Corollary 2.3. Let A be a measurement matrix with RIC 5a^ < — 1- Let x be an 

J ) 2 

s -sparse signal with noisy measurements y = b + e = Ax + e. Let x" be the reconstruction 
from CoSaMP using decoding matrix $ (instead of A). Then the estimation satisfies 



\x*-x\\ 2 <C ■ (e^\\b\\ 2 + \\e\\ 2 



where = \\A - <3?|| 2 ^. 



We now analyze the case of mixed operators and prove our main result, Theorem 12.21 
We will first utilize a result from [13] which states that matrices which are "close" to each 
other also have similar RICs. 

Lemma 2.4 (RIP for $ [13] )• For any s = 1, 2, . . ., assume and fix the RIC S s associated 
with A, and the relative perturbation ej^ associated with A — as defined in h2.1\) . Then 
the RIC constant 5 S for matrix satisfies 

, 2 

'A 



(2.9) 5 S < (1 + Ss) ( 1 + e (s) 



We now prove our main result, Theorem 

Proof of Theorem \2.2\ Lemma [2.41 applied to the case where 64s < — ^ 1 implies that 

(l+e^ >) 2 

the matrix $ has an RIC that satisfies 5^ < 0.1. We can then apply Theorem 11.21 with 
measurements y = <&x + (A — $>)x + e. This implies that the reconstruction x* satisfies 

(2.10) ||x" - x\\ 2 <C ■ [\\x- x s \\ 2 + -pllx - x„||i + \\{A- $)s|| 2 + ||e|| 2 

V \/s 



As seen in Proposition 3.5 of [15], the RIP implies that for an arbitrary signal x, 

\\Ax\\ 2 < \Jl + 5 s \\\x\\ 2 + -^=||a;||i^ . 

As shown in |13j . this and the RIP imply that 

\\Ax\\ 2 > \/l - <U|x s ||2 - \/l + S s (\\x - x 8 \\2 + — 7= [las - x s ||i). 

V s 

We then have that 

(s) 

\\/A I, ^ \\A-$ 2 \\X-Xs 2+ \\A-$ 2 \\ x s 2 ,, „ ,, 

||(A - <S>)x\\ 2 < — -===— H , 1 H iprll^lb- 

Vl - <3 s |f s ||2 - yl + <mIf - x sh + t^If - 

Condition (|2.7|) then gives us 

\\(A - $)x\\ 2 < — i- \Ax\ 2 

gVl ~ Os ||2 



2|U-$|| 2 2p-$n 2 



+ "" " IIAarl 



Applying the inequality S s < 5^ < 0.1 yields 

|| (A - *)x|| 2 < C'(\\A - n 2 a s + \\A - $|| 2 S) ) \\Ax\\ 2 . 
Combined with (|2.10p . this completes the claim. □ 

3. Discussion 

One should of course make sure that the requirements imposed by Theorems 12.11 and 12.21 
are reasonable and make sense. For instance, in Theorem 12.11 to ensure that the RIC 
S 2s > we can set the left-hand side of condition (|2.3|) to zero. Rearranging, this re- 
quires that e^ s ^ < \f2 — 1, which addresses the question "how dissimilar can A and <5 
be?" Loosely phrased, the answer is that the spectral norm of 2s-column submatrices of <5 
cannot deviate by more than about 19% of spectral norm of 2s-column submatrices of A. 
The corresponding condition (|2.8p in Theorem 12.21 requires that e^ s) < vTI - 1, which 
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translates to an approximate 5% dissimilarity between A and <£. The second condition, 
(I2.4p . in Theorem 12.11 is discussed in [13], and essentially requires that the signal be well 
approximated by a sparse signal. This is, of course, a standard assumption in compressed 
sensing. The same argument holds for condition (|2.7p in Theorem 12.21 

In conclusion, real-world applications often utilize different operators (perhaps unknow- 
ingly) to encode and decode a signal. The perturbation of the sensing matrix creates 
multiplicative noise in the system. This type of noise is fundamentally different than simple 
additive noise. For example, to overcome a poor signal-to-noise ratio (SNR) due to addi- 
tive noise, one would typically increase the strength of the signal. However, if the noise is 
multiplicative this will not improve the situation, and in fact will actually cause the error 
to grow. Thus the impact on reconstruction from the error in the sensing matrices needs to 
be analyzed. Our Theorems 12.11 and 12.21 do just that. They show the effect of using mixed 
operators to recover a signal in compressed sensing: the stability of the recovered signal is 
a linear function of the operator perturbations defined above. This work confirms that this 
is the case both for ^i-minimization and CoSaMP. These results can easily be extended to 
other greedy algorithms as well. 
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